High-performance Implementation Research Articles

SMaLL: Software for Rapidly Instantiating Machine Learning Libraries

Interest in deploying deep neural network (DNN) inference on edge devices has resulted in an explosion of the number and types of hardware platforms that machine learning (ML) libraries must support. High-level programming interfaces, such as TensorFlow, can be readily ported across different devices; however, maintaining performance when porting the low-level implementation is more nuanced. High-performance inference implementations require an effective mapping of the high-level interface to the target hardware platform. Commonly, this mapping may use optimizing compilers to generate code at compile time or high-performance vendor libraries that have been specialized to the target platform. Both approaches rely on expert knowledge across levels to produce an efficient mapping. This makes supporting new architectures difficult and time-consuming. In this work, we present a DNN library framework, SMaLL, that is easily extensible to new architectures. The framework uses a unified loop structure and shared, cache-friendly data format across all intermediate layers, eliminating the time and memory overheads incurred by data transformation between layers. Each layer is implemented by specifying its dimensions and a kernel , the key computing operation of that layer. The unified loop structure and kernel abstraction allows the reuse of code across layers and computing platforms. New architectures only require a few hundred lines in the kernel to be redesigned. To show the benefits of our approach, we have developed software that supports a range of layer types and computing platforms; this software is easily extensible for rapidly instantiating high-performance DNN libraries. An evaluation of the portability of our framework is shown by instantiating end-to-end networks from the MLPerf:tiny benchmark suite on five ARM platforms and one x86 platform (an AMD Zen 2). We also show that the end-to-end performance is comparable to or better than ML frameworks such as TensorFlow, TVM, and LibTorch.

An Application-Driven Method for Assembling Numerical Schemes for the Solution of Complex Multiphysics Problems

Within recent years, considerable progress has been made regarding high-performance solvers for partial differential equations (PDEs), yielding potential gains in efficiency compared to industry standard tools. However, the latter largely remains the status quo for scientists and engineers focusing on applying simulation tools to specific problems in practice. We attribute this growing technical gap to the increasing complexity and knowledge required to pick and assemble state-of-the-art methods. Thus, with this work, we initiate an effort to build a common taxonomy for the most popular grid-based approximation schemes to draw comparisons regarding accuracy and computational efficiency. We then build upon this foundation and introduce a method to systematically guide an application expert through classifying a given PDE problem setting and identifying a suitable numerical scheme. Great care is taken to ensure that making a choice this way is unambiguous, i.e., the goal is to obtain a clear and reproducible recommendation. Our method not only helps to identify and assemble suitable schemes but enables the unique combination of multiple methods on a per-field basis. We demonstrate this process and its effectiveness using different model problems, each comparing the resulting numerical scheme from our method with the next best choice. For both the Allen–Cahn and advection equations, we show that substantial computational gains can be attained for the recommended numerical methods regarding accuracy and efficiency. Lastly, we outline how one can systematically analyze and classify a coupled multiphysics problem of considerable complexity with six different unknown quantities, yielding an efficient, mixed discretization that in configuration compares well to high-performance implementations from the literature.

Leveraging index compression techniques to optimize the use of co-processors


 
 
 The significant presence that many-core devices like GPUs have these days, and their enormous computational power, motivates the study of sparse matrix operations in this hardware. The essential sparse kernels in scientific computing, such as the sparse matrix-vector multiplication (SpMV), usually have many different high-performance GPU implementations. Sparse matrix problems typically imply memory-bound operations, and this characteristic is particularly limiting in massively parallel processors. This work revisits the main ideas about reducing the volume of data required by sparse storage formats and advances in understanding some compression techniques. In particular, we study the use of index compression combined with sparse matrix reordering techniques in CSR and explore other approaches using a blocked format. The systematic experimental evaluation on a large set of real-world matrices confirms that this approach achieves meaningful data storage reductions. Additionally, we find promising results of the impact of the storage reduction on the execution time when using accelerators to perform the mathematical kernels.
 
 

A Configurable CRYSTALS-Kyber Hardware Implementation with Side-Channel Protection

In this work, we present a configurable and side channel resistant implementation of the post-quantum key-exchange algorithm CRYSTALS-Kyber . The implemented design can be configured for different performance and area requirements leading to different trade-offs for different applications. A low area implementation can be achieved in 5,269 LUTs and 2,422 FFs, whereas a high performance implementation required 7,151 LUTs and 3,730 FFs. Due to a deeply pipelined architecture, a high operating speed of more than 250 MHz could be achieved on 28nm Xilinx FPGAs. The side channel resistance is implemented using a carefully chosen set of novel and known techniques such as Fault Detection Hashes, Instruction Randomization, FSM Protection and so on. resulting in a low overhead of less than 5% while being highly configurable. To the best of our knowledge, this work presents the first side-channel and fault attack protected configurable accelerator for CRYSTALS-Kyber . Using TVLA (test vector leakage assessment), we validate the implemented protection techniques and demonstrate that the design does not leak information even after 200 K traces. Furthermore, one of the configuration choices results in the smallest hardware implementation of CRYSTALS-Kyber known in the literature.

High-Performance Hardware Implementation of MPCitH and Picnic3

Picnic is a post-quantum digital signature, the security of which relies solely on symmetric-key primitives such as block ciphers and hash functions instead of number theoretic assumptions. One of the main concerns of Picnic is the large signature size. Although Katz et al.’s protocol (MPCitH-PP) significantly reduces the size of Picnic, the involvement of more parties in MPCitH-PP leads to longer signing/verification times and more hardware resources. This poses new challenges for implementing high-performance Picnic on resource-constrained FPGAs. So far as we know, current works on the hardware implementation of MPCitH-based signatures are compatible with 3 parties only. In this work, we investigate the optimization of the implementation of MPCitH-PP and successfully deploying MPCitH-PP with more than three parties on resource-constrained FPGAs, e.g., Xilinx Artix-7 and Kintex-7, for the first time. In particular, we propose a series of optimizations, which include pipelining and parallel optimization for MPCitH-PP and the optimization of the underlying symmetric primitives. Besides, we make a slight modification to the computation of the offline commitment, which can further reduce the number of computations of Keccak. These optimizations significantly improve the hardware performance of Picnic3. Signing messages on our FPGA takes 0.047 ms for the L1 security level, outperforming Picnic1 with hardware by a factor of about 5.3, which is the fastest implementation of post-quantum signatures as far as we know. Our FPGA implementation for the L5 security level takes 0.146 ms beating Picnic1 by a factor of 8.5, and outperforming Sphincs by a factor of 17.3.

High-Performance Implementation of a 1024-bit Full-Word Montgomery Modular Multiplier for High-Speed Encryption Hardware Circuit

High-Performance Implementation of a 1024-bit Full-Word Montgomery Modular Multiplier for High-Speed Encryption Hardware Circuit

High-performance unified modular multiplication algorithm and hardware architecture over G(2m)

The finite field Modular Multiplier (MM) over GF (2m) based on the National Institute of Standards and Technology (NIST) recommended polynomials can be used as a critical component in many cryptosystems. But much of the research suffers from the problems of only supporting a single curve or being very inefficient. In this paper, we present a KA-based unified algorithm (hybrid field size) and its corresponding architecture for high-performance implementation of the systolic modular multiplier over GF (2m), which can support all the NIST recommended polynomials. A number of efficient techniques have been explored and used to realize a high-performance implementation of the multiplier. First, we propose a three-term KA based MM approach and make it fit for systolic implementation. Second, we propose a configurable, unified irreducible polynomial that can support all five curves recommended by NIST and the corresponding modular reduction method. Third, the corresponding systolic structure is also designed in accordance with (1) and (2) to improve efficiency, and we adopt register sharing and computation unit sharing methods to reduce the space complexity of the proposed structure. Additionally, parallel computation of two163-width MMs is employed to improve resource utilization further. From the synthesis results on ASIC, it is shown that the proposed multipliers have significantly lower latency and higher performance than the existing designs. e.g., the proposed structure could achieve up to a 73 % reduction in area-delay product (ADP) over the existing structures.

A fast and efficient computation method for reflective diffraction simulations

We present a new computation method for simulating reflection high-energy electron diffraction and the total-reflection high-energy positron diffraction experiments. The two experiments are used commonly for the structural analysis of material surface. The present paper improves the conventional numerical method, the multi-slice method, for faster computation, since the present method avoids the matrix-eigenvalue solver for the computation of matrix exponentials and can adopt higher-order ordinary differential equation solvers. Moreover, we propose a high-performance implementation based on multi-thread parallelization and cache-reusable subroutines. In our tests, this new method performs up to 2,000 times faster than the conventional method without compromising accuracy.

Mojo: A Python-based Language for High-Performance AI Models and Deployment

Python has become a popular language for AI model development due to its elegant and flexible programming capabilities, extensive tool ecosystem, and high-performance libraries like Numpy and PyTorch. However, Python's execution speed remains a challenge, especially for performance-critical inner loops. To address this, Python programmers often rely on wrappers for C, FORTRAN, or Rust code, leading to a "two-language" approach that introduces complexities in deployment and debugging. This research paper introduces Mojo, a promising solution to the Python performance issue, which is essentially Python++ and built on top of MLIR (Multi-Level Intermediate Representation). Mojo is a rigorously designed superset of Python that allows seamless integration of high-performance implementations by switching to a faster "mode." This paper discusses the key features of Mojo, its deployment advantages, and its comparison with other alternatives in the AI and ML development landscape.

A Shifting Filter Framework for Dynamic Set Queries

Set query is a fundamental problem in computer systems. Plenty of applications rely on the query results of membership, association, and multiplicity. A traditional method that addresses such a fundamental problem is derived from Bloom filter. However, such methods may fail to support element deletion, require additional filters or apriori knowledge, making them unamenable to a high-performance implementation for dynamic set representation and query. In this paper, we envision a novel sketch framework that is multi-functional, non-parametric, space efficient, and deletable. As far as we know, none of the existing designs can guarantee such features simultaneously. To this end, we present a general shifting framework to represent auxiliary information (such as multiplicity, association) with the offset. Thereafter, we specify such design philosophy for a hash table horizontally at the slot level, as well as vertically at the bucket level. Theoretical and experimental results jointly demonstrate that our design works exceptionally well with three types of set queries under small memory.

A High-Performance and Cost-Effective Field Programmable Gate Array-Based Motor Drive Emulator.

This work presents a hardware-based digital emulator capable of digitally driving a permanent magnet synchronous machine electronic setup. The aim of this work is to present a high-performance, cost-effective, and portable complementary solution when new paradigms of electronic drive design are generated, such as machine early failure detection, fault-tolerant drive, and high-performance control strategy implementations. In order to achieve the high performance required by the digital emulator, the electronic drive models (permanent-magnet synchronous machine, voltage-source inverter, motor-control strategy) are digitally described in Verilog hardware description language and implemented on a field programmable gate array (FPGA) digital platform using two approaches: parallel and sequential methods. The results obtained show the effectiveness of the digital emulator design, and the resources used by the solution presented can be implemented on a low-cost digital platform that reveals a cost-effective operation of the solution presented.

Topgun: An ECC Accelerator for Private Set Intersection

Elliptic Curve Cryptography (ECC), one of the most widely used asymmetric cryptographic algorithms, has been deployed in Transport Layer Security (TLS) protocol, blockchain, secure multiparty computation, and so on. As one of the most secure ECC curves, Curve25519 is employed by some secure protocols, such as TLS 1.3 and Diffie-Hellman Private Set Intersection (DH-PSI) protocol. High-performance implementation of ECC is required, especially for the DH-PSI protocol used in privacy-preserving platform. Point multiplication, the chief cryptographic primitive in ECC, is computationally expensive. To improve the performance of DH-PSI protocol, we propose Topgun, a novel and high-performance hardware architecture for point multiplication over Curve25519. The proposed architecture features a pipelined Finite-field Arithmetic Unit and a simple and highly efficient instruction set architecture. Compared to the best existing work on Xilinx Zynq 7000 series FPGA, our implementation with one Processing Element can achieve 3.14× speedup on the same device. To the best of our knowledge, our implementation appears to be the fastest among the state-of-the-art works. We also have implemented our architecture consisting of 4 Compute Groups, each with 16 PEs, on an Intel Agilex AGF027 FPGA. The measured performance of 4.48 Mops/s is achieved at the cost of 86 Watts power, which is the record-setting performance for point multiplication over Curve25519 on FPGAs.

FireFly: A High-Throughput Hardware Accelerator for Spiking Neural Networks With Efficient DSP and Memory Optimization

Spiking neural networks (SNNs) have been widely used due to their strong biological interpretability and high energy efficiency. With the introduction of the backpropagation algorithm and surrogate gradient, the structure of spiking neural networks has become more complex, and the performance gap with artificial neural networks has gradually decreased. However, most SNN hardware implementations for field-programmable gate arrays (FPGAs) cannot meet arithmetic or memory efficiency requirements, which significantly restricts the development of SNNs. They do not delve into the arithmetic operations between the binary spikes and synaptic weights or assume unlimited on-chip RAM resources by using overly expensive devices on small tasks. To improve arithmetic efficiency, we analyze the neural dynamics of spiking neurons, generalize the SNN arithmetic operation to the multiplex-accumulate operation, and propose a high-performance implementation of such operation by utilizing the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory efficiency, we design a memory system to enable efficient synaptic weights and membrane voltage memory access with reasonable on-chip RAM consumption. Combining the above two improvements, we propose an FPGA accelerator that can process spikes generated by the firing neuron on-the-fly (FireFly). FireFly is the first SNN accelerator that incorporates DSP optimization techniques into SNN synaptic operations. FireFly is implemented on several FPGA edge devices with limited resources but still guarantees a peak performance of 5.53TOP/s at 300MHz. As a lightweight accelerator, FireFly achieves the highest computational density efficiency compared with existing research using large FPGA devices.

Dual-band optical collimator based on deep-learning designed, fabrication-friendly metasurfaces

Abstract Metasurfaces, which consist of arrays of ultrathin planar nanostructures (also known as “meta-atoms”), offer immense potential for use in high-performance optical devices through the precise manipulation of electromagnetic waves with subwavelength spatial resolution. However, designing meta-atom structures that simultaneously meet multiple functional requirements (e.g., for multiband or multiangle operation) is an arduous task that poses a significant design burden. Therefore, it is essential to establish a robust method for producing intricate meta-atom structures as functional devices. To address this issue, we developed a rapid construction method for a multifunctional and fabrication-friendly meta-atom library using deep neural networks coupled with a meta-atom selector that accounts for realistic fabrication constraints. To validate the proposed method, we successfully applied the approach to experimentally demonstrate a dual-band metasurface collimator based on complex free-form meta-atoms. Our results qualify the proposed method as an efficient and reliable solution for designing complex meta-atom structures in high-performance optical device implementations.

A high-performance implementation of atomistic spin dynamics simulations on x86 CPUs

Atomistic spin dynamics simulations provide valuable information about the energy spectrum of magnetic materials in different phases, allowing one to identify instabilities and the nature of their excitations. However, the time cost of evaluating the spin-spin correlation functions in real space increases quadratically as the number of spins N, leading to significant computational effort, making the simulation of large spin systems very challenging. In this work, we propose to use a highly optimized general matrix multiply (GEMM) subroutine to calculate the dynamical spin-spin correlation function that can achieve near-optimal hardware utilization. Furthermore, we fuse the element-wise operations in the calculation of S(q,t) into the in-house GEMM kernel, which results in further performance improvements of 44% - 71% on several relatively large lattice sizes when compared to the implementation that uses the GEMM subroutine in OpenBLAS, which is the state-of-the-art open source library for Basic Linear Algebra Subroutine (BLAS).

A Framework for Deep Learning Emulation of Numerical Models With a Case Study in Satellite Remote Sensing.

Numerical models based on physics represent the state of the art in Earth system modeling and comprise our best tools for generating insights and predictions. Despite rapid growth in computational power, the perceived need for higher model resolutions overwhelms the latest generation computers, reducing the ability of modelers to generate simulations for understanding parameter sensitivities and characterizing variability and uncertainty. Thus, surrogate models are often developed to capture the essential attributes of the full-blown numerical models. Recent successes of machine learning methods, especially deep learning (DL), across many disciplines offer the possibility that complex nonlinear connectionist representations may be able to capture the underlying complex structures and nonlinear processes in Earth systems. A difficult test for DL-based emulation, which refers to function approximation of numerical models, is to understand whether they can be comparable to traditional forms of surrogate models in terms of computational efficiency while simultaneously reproducing model results in a credible manner. A DL emulation that passes this test may be expected to perform even better than simple models with respect to capturing complex processes and spatiotemporal dependencies. Here, we examine, with a case study in satellite-based remote sensing, the hypothesis that DL approaches can credibly represent the simulations from a surrogate model with comparable computational efficiency. Our results are encouraging in that the DL emulation reproduces the results with acceptable accuracy and often even faster performance. We discuss the broader implications of our results in light of the pace of improvements in high-performance implementations of DL and the growing desire for higher resolution simulations in the Earth sciences.

Health and economic impact of delaying large-scale HPV vaccination and screening implementation on cervical cancer in China: a modelling study

Current uptake of HPV vaccination and screening in China is far below World Health Organization 2030 targets for cervical cancer elimination. We quantified health and economic losses of delaying large-scale HPV vaccination and screening implementation in China. We used a previously validated transmission model to project lifetime health benefits, costs, effectiveness, and timeline for cervical cancer elimination of alternative scenarios, including combining HPV vaccination initiated from 2022 to 2030 with screening in different modalities and coverage increase rates, as well as screening alone. All women living or projected to be born in China during 2022-2100 were considered. We employed a societal perspective. Regardless of vaccine type, immediate large-scale vaccination initiated in 2022 and achieving 70% coverage of HPV-based screening in 2030 (no-delay scenario) would be the least costly and most effective. Compared with the no-delay scenario, delaying vaccination by eight years would result in 434,000-543,000 additional cervical cancer cases, 138,000-178,000 deaths, and $2863-4437 million costs, and delay elimination by 9-10 years. Even with immediate vaccination, the gradual scale-up of LBC-based screening to 70% coverage in 2070 would result in 2,530,000-3,060,000 additional cases, 909,000-1,040,000 deaths, and $5098-5714 million costs compared with no-delay scenario, and could not achieve elimination if domestic 2vHPV or 4vHPV vaccines are used (4.09-4.21 cases per 100,000 woman in 2100). Delaying large-scale HPV vaccination and/or high-performance screening implementation has detrimental consequences for cervical cancer morbidity, mortality, and expenditure. These findings should spur health authorities to expedite large-scale vaccine rollout and improve screening. Bill & Melinda Gates Foundation (INV-031449 and INV-003174) and CAMS Innovation Fund for Medical Sciences (CIFMS) (2021-I2M-1-004).

Advances in Simulating the Global Spatial Heterogeneity of Air Quality and Source Sector Contributions: Insights into the Global South.

High-resolution simulations are essential to resolve fine-scale air pollution patterns due to localized emissions, nonlinear chemical feedbacks, and complex meteorology. However, high-resolution global simulations of air quality remain rare, especially of the Global South. Here, we exploit recent developments to the GEOS-Chem model in its high-performance implementation to conduct 1-year simulations in 2015 at cubed-sphere C360 (∼25 km) and C48 (∼200 km) resolutions. We investigate the resolution dependence of population exposure and sectoral contributions to surface fine particulate matter (PM2.5) and nitrogen dioxide (NO2), focusing on understudied regions. Our results indicate pronounced spatial heterogeneity at high resolution (C360) with large global population-weighted normalized root-mean-square difference (PW-NRMSD) across resolutions for primary (62-126%) and secondary (26-35%) PM2.5 species. Developing regions are more sensitive to spatial resolution resulting from sparse pollution hotspots, with PW-NRMSD for PM2.5 in the Global South (33%), 1.3 times higher than globally. The PW-NRMSD for PM2.5 for discrete southern cities (49%) is substantially higher than for more clustered northern cities (28%). We find that the relative order of sectoral contributions to population exposure depends on simulation resolution, with implications for location-specific air pollution control strategies.

Training End-to-End Unrolled Iterative Neural Networks for SPECT Image Reconstruction.

Training end-to-end unrolled iterative neural networks for SPECT image reconstruction requires a memory-efficient forward-backward projector for efficient backpropagation. This paper describes an open-source, high performance Julia implementation of a SPECT forward-backward projector that supports memory-efficient backpropagation with an exact adjoint. Our Julia projector uses only ~5% of the memory of an existing Matlab-based projector. We compare unrolling a CNN-regularized expectation-maximization (EM) algorithm with end-to-end training using our Julia projector with other training methods such as gradient truncation (ignoring gradients involving the projector) and sequential training, using XCAT phantoms and virtual patient (VP) phantoms generated from SIMIND Monte Carlo (MC) simulations. Simulation results with two different radionuclides (90Y and 177Lu) show that: 1) For 177Lu XCAT phantoms and 90Y VP phantoms, training unrolled EM algorithm in end-to-end fashion with our Julia projector yields the best reconstruction quality compared to other training methods and OSEM, both qualitatively and quantitatively. For VP phantoms with 177Lu radionuclide, the reconstructed images using end-to-end training are in higher quality than using sequential training and OSEM, but are comparable with using gradient truncation. We also find there exists a trade-off between computational cost and reconstruction accuracy for different training methods. End-to-end training has the highest accuracy because the correct gradient is used in backpropagation; sequential training yields worse reconstruction accuracy, but is significantly faster and uses much less memory.

COPMA: Compact and Optimized Polynomial Multiplier Accelerator for High-Performance Implementation of LWR-Based PQC

The rapid progress in quantum computing has initiated a new round of cryptographic innovation, that is, developing postquantum cryptography (PQC) to resist attacks from well-established quantum computers. In this brief, we propose a novel compact and optimized polynomial multiplier accelerator (COPMA) for high-performance implementation of learning-with-rounding (LWR)-based PQC. As not many LWR-based PQC schemes are available in the literature, we have just used Saber, the National Institute of Standards and Technology (NIST) third-round PQC standardization finalist, as a typical case study example. First of all, we have formulated the polynomial multiplication, the major component of Saber, into a novel “subpolynomial”-based processing format for compact computation (yet has the potential for fast operation). Then, we have designed the proposed algorithm into an area-efficient polynomial multiplication hardware accelerator with high-frequency operational capability. Finally, we have verified the efficiency of the developed COPMA and have deployed it to build a cryptoprocessor. The implementation and analysis demonstrate the superior performance of the proposed COPMA. The proposed strategy is highly efficient and can be extended to build other PQC hardware accelerators.